Shielded Planning Guided Data-Efficient and Safe Reinforcement Learning.

Safe reinforcement learning (RL) has shown great potential for building safe general-purpose robotic systems. While many existing works have focused on post-training policy safety, it remains an open problem to ensure safety during training as well as to improve exploration efficiency. Motivated to address these challenges, this work develops shielded planning guided policy optimization (SPPO), a new model-based safe RL method that augments policy optimization algorithms with path planning and shielding mechanism. In particular, SPPO is equipped with shielded planning for guided exploration and efficient data collection via model predictive path integral (MPPI), along with an advantage-based shielding rule to keep the above processes safe. Based on the collected safe data, a task-oriented parameter optimization (TOPO) method is used for policy improvement, as well as the observation-independent latent dynamics enhancement. In addition, SPPO provides explicit theoretical guarantees, i.e., clear theoretical bounds for training safety, deployment safety, and the learned policy performance. Experiments demonstrate that SPPO outperforms baselines in terms of policy performance, learning efficiency, and safety performance during training.

Data-Driven Safe Policy Optimization for Black-Box Dynamical Systems With Temporal Logic Specifications.

Learning-based policy optimization methods have shown great potential for building general-purpose control systems. However, existing methods still struggle to achieve complex task objectives while ensuring policy safety during learning and execution phases for black-box systems. To address these challenges, we develop data-driven safe policy optimization (D 2 SPO), a novel reinforcement learning (RL)-based policy improvement method that jointly learns a control barrier function (CBF) for system safety and a linear temporal logic (LTL) guided RL algorithm for complex task objectives. Unlike many existing works that assume known system dynamics, by carefully constructing the data sets and redesigning the loss functions of D 2 SPO, a provably safe CBF is learned for black-box dynamical systems, which continuously evolves for improved system safety as RL interacts with the environment. To deal with complex task objectives, we take advantage of the capability of LTL in representing the task progress and develop LTL-guided RL policy for efficient completion of various tasks with LTL objectives. Extensive numerical and experimental studies demonstrate that D 2 SPO outperforms most state-of-the-art (SOTA) baselines and can achieve over 95% safety rate and nearly 100% task completion rates. The experiment video is available at https://youtu.be/2RgaH-zcmkY.

Task-Driven Reinforcement Learning With Action Primitives for Long-Horizon Manipulation Skills.

It is an interesting open problem to enable robots to efficiently and effectively learn long-horizon manipulation skills. Motivated to augment robot learning via more effective exploration, this work develops task-driven reinforcement learning with action primitives (TRAPs), a new manipulation skill learning framework that augments standard reinforcement learning algorithms with formal methods and parameterized action space (PAS). In particular, TRAPs uses linear temporal logic (LTL) to specify complex manipulation skills. LTL progression, a semantics-preserving rewriting operation, is then used to decompose the training task at an abstract level, informs the robot about their current task progress, and guides them via reward functions. The PAS, a predefined library of heterogeneous action primitives, further improves the efficiency of robot exploration. We highlight that TRAPs augments the learning of manipulation skills in both learning efficiency and effectiveness (i.e., task constraints). Extensive empirical studies demonstrate that TRAPs outperforms most existing methods.Sign.

Safe reinforcement learning under temporal logic with reward design and quantum action selection.

This paper proposes an advanced Reinforcement Learning (RL) method, incorporating reward-shaping, safety value functions, and a quantum action selection algorithm. The method is model-free and can synthesize a finite policy that maximizes the probability of satisfying a complex task. Although RL is a promising approach, it suffers from unsafe traps and sparse rewards and becomes impractical when applied to real-world problems. To improve safety during training, we introduce a concept of safety values, which results in a model-based adaptive scenario due to online updates of transition probabilities. On the other hand, a high-level complex task is usually formulated via formal languages, including Linear Temporal Logic (LTL). Another novelty of this work is using an Embedded Limit-Deterministic Generalized Büchi Automaton (E-LDGBA) to represent an LTL formula. The obtained deterministic policy can generalize the tasks over infinite and finite horizons. We design an automaton-based reward, and the theoretical analysis shows that an agent can accomplish task specifications with the maximum probability by following the optimal policy. Furthermore, a reward shaping process is developed to avoid sparse rewards and enforce the RL convergence while keeping the optimal policies invariant. In addition, inspired by quantum computing, we propose a quantum action selection algorithm to replace the existing [Formula: see text]-greedy algorithm for the balance of exploration and exploitation during learning. Simulations demonstrate how the proposed framework can achieve good performance by dramatically reducing the times to visit unsafe states while converging optimal policies.

Divergent Component of Motion Planning and Adaptive Repetitive Control for Wearable Walking Exoskeletons.

Wearable walking exoskeletons show great potentials in helping patients with neuro musculoskeletal stroke. Key to the successful applications is the design of effective walking trajectories that enable smooth walking for exoskeletons. This work proposes a walking planning method based on the divergent component of motion to obtain a stable joint angle trajectory. Since periodic and nonperiodic disturbances are ubiquitous in the repeating walking motion of an exoskeleton system, a major challenge in the walking control of wearable exoskeleton is the joint angle drift problem, that is, the joint angle motion trajectories are not necessarily periodic due to the presence of disturbance. To address this challenge, this work develops an adaptive repetitive control strategy to guarantee that the motion trajectories of joint angle are repetitive. In particular, by treating the disturbance as system uncertainties, an adaptive controller is designed to compensate for the uncertainties based on an integral-type Lyapunov function. A fully saturated learning approach is then developed to achieve asymptotic tracking of repetitive walking trajectories. Extensive experiments are carried out to demonstrate the effectiveness of the tracking performance.

Molecular prevalence of Tetratrichomonas gallinarum and Trichomonas gallinae in three domestic free-range poultry breeds in Anhui Province, China.

Tetratrichomonas gallinarum and Trichomonas gallinae can colonize the alimentary tract of domestic birds. However, little information is available on the epidemiology of the two trichomonad species in domestic free-range poultry in China. In this study, the occurrence and genetic characteristic of T. gallinarum and T. gallinae among free-range chickens, ducks, and geese in Anhui Province, China, were investigated. The 1910 fecal samples collected from 18 free-range poultry farms throughout Anhui Province were examined for the presence of T. gallinarum and T. gallinae by PCR and sequence analysis of the small subunit (SSU) rRNA gene of T. gallinarum and ITS1-5.8S-ITS2 sequence of T. gallinae. The overall occurrence of T. gallinarum in poultry was 1.2% (22/1910), with infection rates of 2.1% (17/829) in chickens, 0.2% (1/487) in ducks, and 0.7% (4/594) in geese. The constructed phylogeny tree using the concatenated ITS1-5.8S-ITS2 region and SSU rRNA indicated the T. gallinarum isolates detected in this study were closely related to previously defined genogroups A, D, and E, respectively. Nine (0.5%) fecal samples were positive for T. gallinae, with infection rates of 0.8% (7/829) in chickens, 0.4% (2/487) in ducks, and 0% (0/594) in geese. Sequence and phylogenetic analysis showed that four T. gallinae ITS1-5.8S-ITS2 sequences obtained from chicken feces and one duck fecal sample belonged to genotype ITS-OBT-Tg-1. This is the first report of the prevalence and genetic characterization of T. gallinarum and T. gallinae in free-range chickens, ducks, and geese in China.

Asymmetric Cooperation Control of Dual-Arm Exoskeletons Using Human Collaborative Manipulation Models.

The exoskeleton is mainly used by subjects who suffer muscle injury to enhance motor ability in the daily life environment. Previous research seldom considers extending human collaboration skills to human-robot collaborations. In this article, two models, that is: 1) the following the better model and 2) the interpersonal goal integration model, are designed to facilitate the human-human collaborative manipulation in tracking a moving target. Integrated with dual-arm exoskeletons, these two models can enable the robot to successfully perform target tracking with two human partners. Specifically, the manipulation workspace of the human-exoskeleton system is divided into a human region and a robot region. In the human region, the human acts as the leader during cooperation, while, in the robot region, the robot takes the leading role. A novel region-based Barrier Lyapunov function (BLF) is then designed to handle the change of leader roles between the human and the robot and ensures the operation within the constrained human and robot regions when driving the dual-arm exoskeleton to track the moving target. The designed adaptive controller ensures the convergence of tracking errors in the presence of region switches. Experiments are performed on the dual-arm robotic exoskeleton for the subject with muscle damage or some degree of motor dysfunctions to evaluate the proposed controller in tracking a moving target, and the experimental results demonstrate the effectiveness of the developed control.

Molecular prevalence and characterization of Cryptosporidium in domestic free-range poultry in Anhui Province, China.

Free-range chickens might mediate the spread of Cryptosporidium oocysts to humans and other animals. Few studies have evaluated the prevalence of Cryptosporidium species in domestic free-range poultry in China. Here, we characterized the prevalence and distribution of species and genotypes of Cryptosporidium in domestic free-range chickens, ducks, and geese in Anhui Province, China. A total of 1910 fresh fecal samples from three poultry species were examined from 18 free-range poultry farms by nested PCR and analysis of the Cryptosporidium SSU rRNA gene. The overall prevalence of Cryptosporidium species was 2.9% (55/1910), with infection rates of 1.3% (11/829) in chickens, 7.3% (36/487) in ducks, and 1.4% (8/594) in geese. C. baileyi (0.6%), C. meleagridis (0.2%), C. galli (0.2%), and C. xiaoi-like genotype (0.2%) were identified in chickens, and only C. baileyi was identified in ducks and geese, with infection rates of 7.4% and 1.3%, respectively. C. baileyi was the most prevalent species. Sequencing of the GP60 gene revealed that the C. meleagridis isolates belonged to the IIIbA26G1R1b subtype. This is the first study to document C. galli and C. xiaoi-like genotype in domestic free-range chickens in China. These findings expand the range of avian hosts known for Cryptosporidium and highlight the need for additional studies to characterize the diversity of Cryptosporidium in avian species.

Controllability Ensured Leader Group Selection on Signed Multiagent Networks.

Leader-follower controllability on signed multiagent networks is investigated in this paper. Specifically, we consider a dynamic signed multiagent network, where the agents interact via neighbor-based Laplacian feedback and the network allows positive and negative edges to capture cooperative and competitive interactions among agents. The agents are classified as either leaders or followers, thus forming a leader-follower signed network. To enable full control of the leader-follower signed network, controllability ensured leader group selection approaches are investigated in this paper, that is, identifying a small subset of nodes in the signed network, such that the selected nodes are able to drive the network to a desired behavior, even in the presence of antagonistic interactions. In particular, graphical characterizations of the controllability of signed networks are first developed based on the investigation of the interaction between network topology and agent dynamics. Since signed path and cycle graphs are basic building blocks for a variety of networks, the developed topological characterizations are then exploited to develop leader selection methods for signed path and cycle graphs to ensure leader-follower controllability. Along with illustrative examples, heuristic algorithms are also developed showing how leader selection methods developed for path and cycle graphs can be potentially extended to more general signed networks. In contrast to existing results that mainly focus on unsigned networks, this paper characterizes controllability and develops leader selection methods for signed networks. In addition, the developed results are generic, in the sense that they are not only applicable to signed networks but also to unsigned networks, since unsigned networks are a particular case of signed networks that only contain positive edges.

Reference Trajectory Reshaping Optimization and Control of Robotic Exoskeletons for Human-Robot Co-Manipulation.

For human-robot co-manipulation by robotic exoskeletons, the interaction forces provide a communication channel through which the human and the robot can coordinate their actions. In this article, an optimization approach for reshaping the physical interactive trajectory is presented in the co-manipulation tasks, which combines impedance control to enable the human to adjust both the desired and the actual trajectories of the robot. Different from previous studies, the proposed method significantly reshapes the desired trajectory during physical human-robot interaction (pHRI) based on force feedback, without requiring constant human guidance. The proposed scheme first formulates a quadratically constrained programming problem, which is then solved by neural dynamics optimization to obtain a smooth and minimal-energy trajectory similar to the natural human movement. Then, we propose an adaptive neural-network controller based on the barrier Lyapunov function (BLF), which enables the robot to handle the uncertain dynamics and the joint space constraints directly. To validate the proposed method, we perform experiments on the exoskeleton robot with human operators for co-manipulation tasks. The experimental results demonstrate that the proposed controller could complete the co-manipulation tasks effectively.

Prevalence of Tetratrichomonas buttreyi and Pentatrichomonas hominis in yellow cattle, dairy cattle, and water buffalo in China.

The trichomonad species Tetratrichomonas buttreyi and Pentatrichomonas hominis have been reported in the bovine digestive tract in only a few studies, and the prevalence and pathogenicity of these two protists in cattle herds remain unknown. In this study, the prevalence of T. buttreyi and P. hominis in yellow cattle, dairy cattle, and water buffalo in Anhui Province, China, was determined with a PCR analysis of the small subunit ribosomal RNA genes. The overall infection rates for T. buttreyi and P. hominis were 8.1% and 5.4%, respectively. Double infections were found in 15 (1.6%) samples from four farms. The prevalence of P. hominis in cattle with abnormal feces was significantly higher than that in cattle with normal feces (χ2 = 13.0, p < 0.01), and the prevalence of T. buttreyi in the northern region of Anhui Province was also significantly higher than that in the mid region (χ2 = 16.6, p < 0.01). Minor allelic variations were detected in the T. buttreyi isolates from cattle in this study, as in other hosts in previous studies. Morphological observations, together with the PCR analysis, demonstrated that the trichomonads isolated in this study were P. hominis. The presence of T. buttreyi and P. hominis indicated that cattle are natural hosts of these two trichomonads and could be a potential source of P. hominis infections in humans and other animal hosts.

A foodborne acute gastroenteritis outbreak caused by GII.P16-GII.2 norovirus in a boarding high school, Beijing, China: a case-control study.

OBJECTIVE: In December 2017, an acute gastroenteritis outbreak involving 61 students occurred in a boarding high school in Beijing, China. We conducted an outbreak investigation immediately in order to determine the cause of this outbreak and provide effective control measures. RESULTS: The laboratory inspection showed that this outbreak was caused by GII.P16-GII.2 norovirus. Risk factor analysis indicated that the lunch provided by Cafeteria 1 on Dec 12 might be the risk factor of the outbreak with an odds ratio (OR) of 3.800 (95% confidence interval [CI] 1.089-13.258). Additionally, a tray line server of Cafeteria 1 was found to have gastro-enteral symptoms recently. Based on the clinical symptoms and epidemiology investigation, the symptomatic server was considered to be the possible source of infection.

Synchronization of Uncertain Euler-Lagrange Systems With Uncertain Time-Varying Communication Delays.

A decentralized controller is designed for leader-based synchronization of communication-delayed networked agents. The agents have heterogeneous dynamics modeled by uncertain, nonlinear Euler-Lagrange equations of motion affected by heterogeneous, unknown, exogenous disturbances. The developed controller requires only one-hop (delayed) communication from network neighbors and the communication delays are assumed to be heterogeneous, uncertain, and time-varying. Each agent uses an estimate of communication delay to provide feedback of estimated recent tracking error. Simulation results are provided to demonstrate the improved performance of the developed controller over other popular control designs.